Arizona Board of Regents: Learner-Centered Education Course Redesign Initiative

Arizona State University

Course Title: Computing and Information Literacy
Redesign Coordinator: Toni Farley

Project Abstract
Final Report (as of 6/1/09)

Project Abstract

The Tempe campus of Arizona State University (ASU) plans to redesign its Computing and Information Literacy course. The traditional course enrolls ~2200 students annually in eight large lecture sections of ~270 students each that meet twice a week.

The traditional course faces a number of problems. It depends too heavily on assignments using commercial software. The material is not learner-centered and does not develop problem-solving skills. Current content is not designed to ensure that students with a broad range of learning styles and levels of preparation will master the content and succeed in the course. This leads to a high DFW rate, among the 30 highest rates at ASU.

ASU will use the Replacement Model in its redesign. The new course will feature learner-centered material focused on problem-solving and will be available in multiple formats to support different learning styles. Problem-solving projects using conceptual models and technology tools, such as XML, scripting, and open-source software, will enable students to interact with peers and solve real-world problems. The traditional format will be replaced with two redesigned formats. The first will be hybrid sections that combine lecture, lab and online resources. There will be one in-class meeting and an open, interactive lab session each week. Lectures will not be mandatory but will be encouraged. Projects and assignments will all be completed online with support provided via a discussion board forum and a lab. The lab sessions will be optional unless students are at risk of failing the course. In that case, the labs will become mandatory until the student exhibits content understanding and is making adequate progress.

The second format will be a fully online section, with a capacity of 500 students, which will replace all in-class meetings with online learning experiences. Students will work with web-based, multi-media resources aligned with the textbook. Automated weekly assessments will provide guided feedback and links to a plethora of online resources. The online option will be available for students who may already have a reasonable amount of computer experience. They will need to pass an online assessment prior to being accepted into the online format.

The redesign will enhance course quality by focusing on problem-solving with the kinds of technology resources the students will continue to use in school and work. Students will use WebQuests to develop critical-thinking and research skills. An advisory board will be created consisting of the redesign team, representatives of academia and industry as well as collaborators from the University of Arizona and Northern Arizona University who have a strong interest in the project. This board will determine the specific content of the course and projects, helping to make sure the course stays current.

Student learning will be assessed by comparing data from two courses taught by the same faculty member before and after the redesign. Student learning outcomes from parallel courses taught by different faculty members during the pilot phase will also be compared. Exam questions will be selected from the traditional course and embedded in the redesigned course exams. DFW rates and student satisfaction surveys will also be analyzed.

The redesigned course will reduce instructional costs each term by 1) replacing four lecture sections with two hybrid sections and one fully online section; 2) increasing section size from 270 to 299, thereby maximizing classroom resources; 3) reducing the number of GTAs from two GTAs each teaching two sections each to one teaching three sections; 4) replacing six undergraduate graders with five undergraduate learning assistants (ULAs) and decreasing the ULA hours; and, 5) replacing two instructors with one faculty coordinator. The cost-per-student will be decreased from $50 to $38, a 24% reduction. The cost savings will be reinvested in redesigning other courses, using this initial redesign as a foundation to increase the quality of all courses across the curriculum. ASU-Tempe’s redesign plan will also be extended to the Downtown, Polytechnic and the West campuses.

Final Report (as of 6/1/09)

Impact on Students

Improved learning

In six prior terms of the course taught in the traditional format, an average of 26% of students earned 70%+ (a C or better.) In the redesigned format, 65% of students earned 70+ (a C or better) in a demonstrably more difficult course.

The redesigned course included a significant shift in content to ensure student preparation in computing and the use of modern tools and techniques to solve real word problems. The overall focus changed to learner-centered problem solving with technology. Content shifted from learning a few basic concepts and common tools, to covering a broader range of technology advances, and familiarity with a wide variety of tools to support future self-learning. Learning was facilitated through a larger number of hands-on projects and assignments, with less focus on memorization and testing.

Improved retention

Retention rates have historically fluctuated in this course. Similar results in the number of students withdrawing from the course remained between traditional and redesigned sections.

Other impacts on students

Student attitudes towards the course improved with the redesign. Using a five-point scale, overall course evaluations averaged 3.60 in traditional sections versus 3.92 in redesigned sections; a 6% increase. Student feedback on the redesigned course was positive and encouraging. When provided an opportunity to give open-ended, blind feedback about the course, many students commented positively on the amount of knowledge and skills they acquired, far in excess of what they had expected.

Impact on Cost Savings

The cost savings plan worked better than expected. Delivering the redesigned course with less support than originally anticipated led to a cost reduction from $50 to $28 per student, a 44% savings. The course was more cost effective due to automation and heavy use of technology tools and the web. Approximately 80% of grading and feedback were automated. The school chose to sustain the model with a higher level of faculty support, which lessened the savings somewhat. The savings are still substantial: a reduction from $50 to $35 per student, which is a 30% savings.

Lessons Learned

Pedagogical Improvement Techniques

  • Challenging content, not procedure. By streamlining course delivery, putting everything online, making full use of online technologies, and clearly defining student responsibilities in a well-organized way, the redesign enabled students to easily follow the flow of the course and keep their focus on the content. Materials were delivered in sequential modules in a standardized format, keeping students moving easily through the course content. Feedback on students’ individual progress was provided continually, with grades easily accessible on the course Blackboard site.
  • Shift to a technology-based course. Other than the textbook, there were no paper-based resources for this course. Projects, assignments and quizzes were all completed using modern teaching technologies and tools. Student evaluations and graded activities incorporated a demonstration of their ability to use such tools. Thus, course delivery was tied to content to a large extent. Learning tools included discussion boards, wikis, screencasts, video demonstrations, automated grading and feedback programs, and interactive tutorials. All content was delivered via a course Blackboard site and organized into weekly modules. Communication and feedback was facilitated through the same site.
  • Emphasis on problem-solving and guided-learning. Course content was delivered in a manner that encouraged students to search for solutions to problems on their own. Guided feedback throughout the course pointed students in the right direction and guided them through the process of finding a solution to a problem rather than simply providing the solution for them. Students were presented with a wide variety of problem-solving tools at an introductory level and provided ample direction to facilitate future self-study.
  • Learner-centered delivery approach. Students were allowed many options for participating in the course. Sections were delivered either fully online or as a hybrid meeting once a week. For each section, 100% of the course was available online. This allowed students enrolled in the hybrid section to decide on a week-by-week basis if they needed to attend lecture for additional help. All students were also given the opportunity to get one-on-one guidance in an open lab staffed by undergraduate learning assistants (ULAs). Student surveys showed a strong preference for receiving guidance from ULAs. Students were also able to communicate with faculty, ULAs and classmates on the course discussion boards which were monitored daily by the instructor. While deadlines were provided for course modules, students were encouraged to work ahead at their own pace to quickly work through the modules they found easy and to spend more time on modules they found challenging.

Cost Savings Techniques

  • Consolidating course management. All sections of the course were managed by a single faculty coordinator and a staff of graduate teaching assistants (GTAs) and ULAs. Under the traditional model, multiple faculty and GTAs administered distinct sections. In the redesign, students in all sections were combined in a single Blackboard site. Allowing online and hybrid sections to share the same Blackboard site provided delivery options for the students, while completely removing redundant work across sections.
  • Automated grading and feedback. The course made use of embedded Blackboard tools and programs developed in-house to automate 80% of the grading and feedback in the course. This significantly reduced the amount of time spent grading and eliminated the need for a large staff of graders that had been necessary under the traditional model. This cost-saving strategy had the benefit of freeing the faculty coordinator to focus on monitoring student progress and freeing the staff to focus on providing assistance to students.

Implementation Issues

  • Technology issues. Midway through the redesign efforts, ASU underwent significant, unexpected technology changes at the university level. These changes completely conflicted with many of the technology tools the team had developed in-house, leading to a need to reinvent many efforts. This took a significant amount of time away from a focus on mid-stream improvements to the course itself. Additionally, the changes exacerbated reporting and evaluation difficulties; success and retention reports changed dramatically in format from the timeframe of the traditional sections to the redesigned sections. Such changes are not anticipated to have impact moving forward, and planned improvements to the course that were not implemented due to these issues are being worked on now.
  • Departmental and faculty buy-in. It was challenging to convince administrators and faculty that the redesign objectives could be met and that the course should be delivered as designed. Challenges in faculty buy-in centered mainly on content issues. Administrators had difficulty in understanding the substantially different way the course would be delivered. They needed to be convinced that it was possible to deliver the course with reduced costs. In fact, full support for the redesign was only achieved after the redesign was complete. The project leader met with administrators to go over the delivery in detail, and a new faculty member was able to deliver the course, thus confirming that the content and design were effective.

Sustainability

A new faculty member was recruited to deliver the redesigned course and has successfully completed a semester. Feedback from the new faculty member is positive. She plans to maintain the course as designed. The transition of the course to the new faculty member was easily facilitated. Alongside the redesigned course, the team created substantial documentation to support faculty, GTA and ULA training. The new faculty member is continuing to implement minor changes in an effort to further improve the quality of the course and to address aspects of the course the team was unable to focus on due to the technical challenges faced. It is certain that the redesigned course will be sustained.

 

Quick Links:

Arizona Board of Regents Main Page...